[Agile] Schema-driven Development
How much design is required when you start a sprint/iteration in Agile? "Big design up front is waste, do just barely good enough just in time." What does this really mean? I would like to share the practice of Agile Modeling worked for me.
Having a well-engineered design is essential for system extension and maintenance, and it is the secret behind most software development success stories. This is especially true when it comes to designing a software component in a collaborative development environment where multiple developers are working together on a project. In addition to designing contracts first with interfaces and abstract classes in object-oriented implementation, special attention needs to be paid toward the data model used between components. They dictate the usability and interoperability of system, and designing them during early phases of the software lifecycle is significant.
As a project manager, having most of the data model and the interface designed up-front (a.k.a. schema-driven) was extremely beneficial on several levels although it was constantly being revised throughout the process. First, it gave developers a bigger picture and a real vision for what other developers were doing and where they were going. Second, it helped them accurately estimate how much of their effort and time it would require to implement the design into a working software. Third, developers have a clearly vision for what changes were required and where the impact areas when changes were made. Finally, it served as a guide to remind us what the software was about as we progressed further into development. It became our benchmark for the initial project scope.
Agile focuses on efficient and effective communication especially between developers. XML technology should be a natural fit for agile developers because it provides a structured syntax that can be used to share information between different kinds of devices, different applications, different teams, and developers in the co-locate team without needing to pass through many layers of conversion. It allows the ability to model contracts with an XML-centric mindset.
XML technology allows developers to set standards defining the information that should appear in a document, and in what structure and data relationship. XML schema (XSD) makes it possible to define the content of a document separately from its definition and structure, making it easy to reuse that content in other applications or for other presentation environments. In general, developers can easily create transformations or queries on XML documents because the data model that the XML represents is itself well known and understood using XML schema. Understand and visualize a system, application, or component.
Use developer language such as XML Schema and minimize documentation. This will understand and visualize a system, application, or component.
In procedural development processes, the data model is itself somewhat indeterminate in most imperative systems, data modeling is a step that realistically only needs to be done later in the process. There is in being able to ascertain ahead of time what specific data model will be required, with the implementation then fleshing out each data model over each iteration.
Even though data model is constantly being revised throughout the process, it is worth to develop schemas that support existing business functionality data transfer and exchange such as “data modeling”, “interface design” and “message design”. The following primary schema types are used in our project:
- Interface Schema
- User Interface
- Functional Interface
- Data Schema
- Built-in or User-defined Types
- Database Schema
- Message Schema
- Simple Object Access Protocol (SOAP)
In Agile incremental development, developers must consider maintain backward compatibility (forward compatibility too if possible) with other interoperating components up-front to prevent the future use of new mechanisms in unintended way. We designed the schema in an iterative and collaborative manner. During each iteration, add just enough components to schema to support the specific user stories that are being implemented. The goal of integration is to maintain compatibility of schema between the incremental. The version control, refactoring and object oriented concept inheritance provides powerful capability for incremental modeling. In case major changes are required as the schema grows, refactored as needed with version control. This approach ensures well-engineered applications with good cohesion throughout the document flow route.
In my experience, it is good to have an architect/technical lead for each project. This person is one of the developers but has additional responsibilities. The key responsibilities of an architect/technical lead are:
- Balancing requirements and technical constraints
- Envisioning requirements and communicating architecture
- Adhering to standards of best practice
- Maintain the data model and the interface
- Agree the technical architecture and ensure the architectural integrity is maintained
- Ensure adherence to standards of best practice
- Verifying implementation
- Combination of mentoring other developers and direct involvement
- management of change and complexity
If using contract-first, we can have a looser coupling between contract and implementation. Such a looser coupling allows us to implement both versions of the contract in one class. We could, for instance, use an XSLT stylesheet to convert any "old-style" messages to the "new-style" messages.
The schema can be tested for quality against an XML Schema Quality of Design Tool (or QoD Tool) and the XMLUnit framework can be helpful in testing the schema as refactor and implement new user stories. We integrated the execution of these validations and tests into our build and continuous integration process.
We integrated the schema modeling effort in our agile project which practices such as user stories, unit test, coding, refactoring, pair programming, common code base, version control, automated build/test and continuous integration.
Schema-driven development requires a different approach but it provides many advantages. The followings are benefits of schema-driven development:
- Adapt changes quickly and increase reusability … auto-generated base codes and add additional properties and functionalities in the derived class.
- Define clear constraints and data relationship … automatic data binding and auto-generated validation logics - validate grammar (structure, form, syntax) and data relationship against a schema.
- Eliminate integration issues and determine impact areas in advance … modulation is increased; once defined, schemas can be reused across multiple components, providing massive reusability.
- Ensure well-engineered and crafted software … information modelling approach; good cohesion between different elements of a distributed component.
- Provide display-independent representation … dynamic, content based page delivery.
- Improve test cases … auto-generated test cases with combinations of the predefined input parameters and optimize the testing time by adding the data frequency attributes.
- Model message … use for structured messaging.
Eventually, this approach helped to automate test by reducing the frequent design changes. Also, layer-based API test approach is seemliness work.
Data validation through auto-generated validation logics filtered out escape and special characters in XML and HTML.
Because of schema-driven development, developers have a much clear view of what other developers are doing and what will be happen during integration. Although it took a little bit longer in design phase, there were no last-minute integration surprises and this approach reduces the overall management burden of many layers of conversion. Eventually it greatly reduces overall development time.
In this writing, I just focused on communications between developers using schema-driven development. To improve communicating with non-technical people, a project manager needs to get the domain knowledge of non-technical people to find ways to understand better. Another challenge as a project manager is how to translate technical language to non-technical language, and vice versa.
Thanks for reading and please feel free to add your comments if you have one.